Robust algorithms for speech reconstruction on mobile devices

This thesis is concerned with reconstructing an intelligible time-domain speech signal from speech recognition features, such as Mel-frequency cepstral coefficients (MFCCs), in a distributed speech recognition(DSR) environment. The initial reconstruction methods in this thesis require, in addition to MFCC vectors, fundamental frequency and voicing information. In the later parts of the thesis these parameters are predicted from the MFCC vectors. Speech reconstruction is achieved by first estimating a spectral envelope from the MFCC vectors. This is combined with excitation information from the fundamental frequency and voicing which enables both a source-filter and sinusoidal model of speech to be investigated and compared. Analysis into the sinusoidal model shows that both clean spectral envelope estimates and robust fundamental frequency estimates are necessary for clean speech reconstruction in noisy environments. Inclusion of spectral subtraction is shown to provide the clean spectral envelope estimates. A comparison of fundamental frequency estimation methods shows the most robust to be obtained from an auditory model. This leads to the proposal of an integrated front-end which replaces the mel-filterbank by the auditory filterbank for MFCC extraction and thereby reduces computation. Speech reconstruction tests reveal that robust fundamental frequency estimation and spectral subtraction lead to intelligible and relatively noise free speech. Evidence in this work has shown that correlation exists between fundamental frequency and the MFCC vectors. This leads to the proposal of predicting the fundamental frequency and voicing of a frame of speech from its MFCC representation. An initial method uses a Gaussian mixture model (GMM) to model the joint density of fundamental frequency and MFCCs. A second method combines the GMMs into a hidden Markov model framework to give a more localized modeling of the joint density. Experimental results on both speaker-dependent and speaker-independent tasks show accurate prediction of the fundamental frequency and voicing from the MFCC vectors which leads to intelligible speech reconstruction.